Metadata services for the Parallax storage system

Parallax is a distributed storage system that uses virtualization to provide storage facilities specifically for virtual environments. In Parallax, fragmentation occurs when the block addresses visible to the guest virtual machine are sequentially placed, but the corresponding physical addresses are not. Because of the copyon-write (CoW) nature of Parallax, as virtual disks are created, cloned, deleted, snapshotted and migrated, some fragmentation of the physical media can occur, potentially incurring seeks even when performing sequential accesses to the virtual disk. As the storage pool ages, performance issues due to unchecked fragmentation, unreclaimed storage space and duplicate data can cause significant concern. CoW snapshots also introduce sharing semantics between virtual disks and snapshots. The ability to create CoW clones of virtual disks from snapshots of other virtual disks leads to more sharing relationships. As a result block reclamation and allocation become non-trivial. We have developed utilities for garbage collecting, de-fragmenting free disk space and virtual disks and reclaiming duplicate read-only blocks in the storage pool managed by Parallax. They work by updating and maintaining the metadata structures related to each virtual disk and its snapshots. They use very coarse grained locking on the metadata and work at the block level. They operate across the storage pool and are agnostic to the operating systems and file systems used by the virtual machines.